Skip to content

Support for Arabic & Category collection #109

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 138 commits into
base: master
Choose a base branch
from
Draft

Support for Arabic & Category collection #109

wants to merge 138 commits into from

Conversation

@ZOUHEIRBN
Copy link

@ZOUHEIRBN ZOUHEIRBN commented Oct 10, 2023

This version of the code is designed for usage with Arabic texts but I believe it can be used with Latin texts as well. It has also support for category collection for purposes such as ontology construction.

ZOUHEIRBN and others added 30 commits October 6, 2023 20:44
@ZOUHEIRBN
Merge pull request suyashb95#86 from jsch8q/patch_pronunciation
bf854b4 
pronunciation parsing for heteronyms
@ZOUHEIRBN
Fetching words from all arabic dialects
1a07dfe
@ZOUHEIRBN
Extracting appendig entities
f19ea48
@ZOUHEIRBN
Collecting language info
3cafcb1
@ZOUHEIRBN
Merge branch 'Suyash458:master' into master
ff6c256
@ZOUHEIRBN
Added saving to database
3346d79
@ZOUHEIRBN
Merge branch 'master' of https://github.com/ZOUHEIRBN/WiktionaryParser
29edfa5
@ZOUHEIRBN
Implemented database word insertion
6bc0078
@ZOUHEIRBN
Added load definitions to database table
ee312be
@ZOUHEIRBN
"Fixed" the definition_appendix insertion issue
52b0cbe
@ZOUHEIRBN
Fixed appendix retrieval
760a206
@ZOUHEIRBN
Sraping related_words (still related_index_issue)
8e86581
@ZOUHEIRBN
Parsing relationships from .nyms class elements
9338d38
@ZOUHEIRBN
Externalized parsing nyms
16320e4
@ZOUHEIRBN
Fixed hashcode matching between relations and defs
5cd3d4d
@ZOUHEIRBN
Enabled saving relationships to database
0760406
@ZOUHEIRBN
Externalized DB connection object
95d791f
@ZOUHEIRBN
Completed saving pipeline
1b48449
@ZOUHEIRBN
Word2Word graph fetching from DB
ff1fe5b
@ZOUHEIRBN
bugfix in relationship parsing (original schema)
cc9a1e0
@ZOUHEIRBN
Removed debug info
4cb5bf2
@ZOUHEIRBN
External processing funcs in 'Collector.save_word'
a630261
@ZOUHEIRBN
Enabled saving related words directly to database
6f988de
@ZOUHEIRBN
Set wordId as FK in relationships table
2ddef8d
@ZOUHEIRBN
Externalized database creation query to file
577ffaf
@ZOUHEIRBN
Added mentions as relationshipType
d35e79d
@ZOUHEIRBN
Fixed inserting mentions to word table
a819833
@ZOUHEIRBN
Fixed null wikiUrls in directly collected words
c2bc870
@ZOUHEIRBN
Externalized saving to DB in collector.save_word
cc81d39
@ZOUHEIRBN
Fixed a bug in appendix retrieval
834d926
ZOUHEIRBN and others added 30 commits January 4, 2024 17:30
@ZOUHEIRBN
Removed deorphanize.py
e70d24d
@ZOUHEIRBN
Merge branch 'master' of https://github.com/ZOUHEIRBN/WiktionaryParser
9958d07
@ZOUHEIRBN
Merge branch 'master' of https://github.com/ZOUHEIRBN/WiktionaryParser
97a139f
@ZOUHEIRBN
Working code but still in doubts
70c4da9
@ZOUHEIRBN
Rewritten phhase 4 deorphanization (BETA)
84ac6f1
@ZOUHEIRBN
Rewritten phhase 4 deorphanization (BETA)
b8b12f9
@benlahmar
Merge branch 'master' of https://github.com/ZOUHEIRBN/WiktionaryParser …
4624ddc 
…into Office-computer
@benlahmar
Merged master branch
9d0cde0
@benlahmar
Added report.py
2aa3afa
@ZOUHEIRBN
Added titles to graph nodes
61895fe
@ZOUHEIRBN
staged changes (NOT STABLE)
c2bea26
@ZOUHEIRBN
Before refactoring changes to relationship colllection in core.py
8ff3bdd
@ZOUHEIRBN
Added def_id to data collection in core.py
84852cf
@ZOUHEIRBN
Changed definition_id generation method
38165fe
@ZOUHEIRBN
Fixed unrelated words issue (BETA)
0ad6c7c
@ZOUHEIRBN
Fixed strange whitespaces in words upon storage
20ab3c0
@ZOUHEIRBN
Fixed unrelated words issue (BETA)
3209139
@ZOUHEIRBN
Fixed bug in collector.py
66086e9
@ZOUHEIRBN
Fixed unrelated words issue (CONFIRMED)
6a402d9
@ZOUHEIRBN
Added parameter for dialect inclusion
fb1efeb
@ZOUHEIRBN
Disabled dialect inclusion in core.py (TEMPORARY)
ce85daf
@ZOUHEIRBN
Fully implemented dialect inclusion in core.py
0526794
@ZOUHEIRBN
Fully implemented dialect inclusion in core.py
1df6ca7
@ZOUHEIRBN
Added filtering to headword-line definitions
8102b2f
@ZOUHEIRBN
Fixed preprocessing bug
560c017
@ZOUHEIRBN
Implemented word magnitude calculation (ALPHA)
cfb9f99
@ZOUHEIRBN
Implemented word magnitude calculation (ALPHA)
23ac94a
@ZOUHEIRBN
Merge remote-tracking branch 'Pulled-repo/Dev'
9ff2554
@ZOUHEIRBN
Added params to export_graph_viz
2c2d420
@ZOUHEIRBN
Implemented def2def (ALPHA)
1ac432c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@ZOUHEIRBN @benlahmar